The present invention relates to estimating and modeling motion of structures or organs in a sequence of medical images, and more particularly to deep learning based modeling and prediction of motion in medical images.
Analyzing and modeling the motion of structures or organs in a sequence of medical images (e.g., cardiac medical images, abdominal medical images, etc.) is an important task in numerous clinical applications, such as image reconstruction, digital subtraction, and organ motion quantification. Having a unique representation of an organ motion can also enable identification of diseases visible through abnormal motion, such as cardiac arrhythmias. Due to these multiple applications, computer-based motion modeling has been the focus of intense research. A major difficult in motion modeling lies in the estimation of organ deformation, as well as the subsequent estimation of a representative motion model. Existing methods typically rely on hand-crafted algorithms which embed strong priors and are therefore not robust and not generalizable to changes of image quality, modality, organs, etc.
Typically, organ motion is studied by finding correspondences between the different frames in an image sequence. Dense correspondences are typically found with deformable registration in which objective functions including a similarity metric between deformed and final images are optimized. Due to the ill-posed nature of the problem, various regularizers are incorporated to add prior knowledge about the transformations under consideration. In order to compute trajectories in a series of frames, diffeomorphic, spatiotemporal B-spline parameterized velocity fields have been introduced. Due to 3D/4D B-spline grids, temporal consistency is taken into account by design. The similarity metric is computed as the sum of the differences between a chose template image and all consecutive frames. On approach proposed the use of barycentric subspaces as a projection space in which motion analysis can be done. Other approaches rely on optical flow to get the dense deformation through the time series, and then manifold learning across a population to learn a mean motion model.
Existing methods for motion modeling in medical imaging rely on time consuming optimization procedures, hand-picked regularizers, and manifold learning on engineered motion features. In cases in which a parameterized motion model is used, the parameterized motion model is constructed manually and typically lacks generalizability. The present inventors have recognized the need for a computer-based medical image motion modeling method that is generalizable to various medical imaging motion modeling tasks and robust to changes in image modality, quality, organs, etc.